The computing industry and the video game industry have seen many changes over the years. As computing power has expanded, developers of video games have created game software adapted to the increased computing power. To this end, video game developers have been coding games that incorporate sophisticated operations and mathematics to produce a very realistic game experience.
These games are presented as part of a gaming system including game consoles, portable game devices, and/or provided as services over a server or the cloud. As is well known, the game console is designed to connect to a monitor (usually a television) and enable user interaction through handheld controllers/input devices. A game console may include specialized processing hardware, including a central processing unit (CPU), a graphics processing unit (GPU) for processing intensive graphics operations, a vector unit for performing geometric transformations, and other glue hardware, firmware, and software. The game console may be further designed with an optical disc tray for receiving game compact discs for local play through the game console. Online and multi-player gaming is also possible, where a user can interactively play against or with other users over the Internet. As game complexity continues to intrigue players, game and hardware manufacturers have continued to innovate to enable additional and more realistic interactivity.
A growing trend in the computer gaming industry is to develop games that increase the interaction between the user and the gaming system. One way of accomplishing a richer interactive experience is to use wireless game controllers whose movement and gestures are tracked by the gaming system. These movements and gestures are used as inputs for the game. Gesture inputs, generally speaking, refer to having an electronic device such as a computing system, video game console, smart appliance, etc., react to some gesture made by the user while playing the game that are captured by the electronic device.
Another way of accomplishing a more immersive interactive experience is to use a head-mounted display (HMD). The HMD is worn by the user and can be configured to present various graphics, such as a view of a virtual space, in a display portion of the HMD. The graphics presented within the HMD can cover a large portion or even all of a user's field of view. Hence, the HMD can provide an immersive experience to the user. As connectivity to the Internet continues to increase, more configurations of HMD systems have been introduced.
The HMD can also be used in a virtual reality system in which a user becomes visually immersed in a computer generated three-dimensional virtual reality scene. In some applications, the entire virtual reality scene as displayed to the user is computer generated. In other applications, a portion of the virtual reality scene is computer generated, with another portion of the virtual reality scene corresponding to video and/or images of real-life objects and/or persons, where such real-life video/images can be rendered in the virtual reality scene in essentially real-time. Such applications may be referred to augmented reality applications.
In various situations, it is necessary to track objects that are visible within the real-world environment associated with the user. For example, it may be necessary to track movements of one or more controller objects, and/or the HMD, and/or other physical objects, including the user. Object tracking can be done by processing images captured by a camera disposed to view the real-world environment associated with the user. It is within this context that the present invention arises.